Mero: Co-Designing an Object Store for Extreme Scale

Within the HPC community, there is consensus that Exascale computing will be plagued with issues related to data I/O performance and data storage infrastructure reliability, caused primarily by the growing gap between compute and storage performance, and the ever increasing volumes of data generated by scientific simulations, instruments and sensors. The architectural assumptions for extreme computing are now changing to accommodate these extreme volumes of data as they transit through scientific workflows. Historically, however, there has been a disconnect between HPC users of extreme-scale storage systems and designers of such systems, as designers make architectural decisions based on an incomplete picture of the use cases they are required to address and users construct individualized workarounds to adjust as necessary. This paper presents the co-design process for deriving an Exascale architecture and presents one such exemplar system: Mero, an object storage software solution specifically architected for BDEC. We systematically gather co-design application requirements from the extreme scale HPC and I/O community and then corroborate those inputs through real world extreme scale use cases, to derive the architecture for Mero.